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ABSTRACT 

In a North Texas school district, students taking online courses in a variety of subject 
areas were not successfully completing courses at the district’s expected levels. The district pays 
for courses and student support while remaining financially responsible to the community and 
programmatically to students’ online success. Students’ unsuccessful completion of these online 
classes, both at the local district and at the state level, became the rationale for the current 
study’s formal evaluation of the program. The qualitative project study evaluated the district’s 
online program and allowed for opportunities for data collection and analysis for improvement 
recommendations. Guided by the constructivist theory, this study examined whether the current 
online instructional program was perceived as effective by students and faculty involved with 
online classes. Qualitative surveys and focus groups were employed to gather data on the overall 
design of the program. Students taking online classes and faculty working with these students 
participated in the survey and randomly selected participants were involved in the focus groups. 
The qualitative data were analyzed utilizing a software program with themes developed and 
interpreted for meaning. Data analysis determined both effective and ineffective practices of the 
program, including issues of local support related to technology and increased 1-on-l support at 
the campus level as identified by students and the need for more resources and familiarity with 
the online course content as identified by mentor teachers. The findings from this research 
provided specific strategies for improving supports that promote success. This increased success 
for high school students will impact social change regarding how administrators, teachers, 
students, and parents view alternative educational opportunities for students in the future. 
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INTRODUCTION 

Budget cuts have forced school districts and higher education institutions to seek 
alternative methods of content delivery for students. Educational opportunities for students 
continue to grow across the United States as technology advances at an exponential rate while 
institutional budgets are continually cut. One alternative method of content delivery possible 
because of technological advancements is online courses or virtual schools. An online and 
blended learning education group that collects data related to K-12 online learning stated, 

“Online learning opportunities have spread into more states than ever before. Keeping Pace now 
counts 45 of the 50 states (plus Washington D.C.) as having a state virtual school or online 
initiative, full-time online schools, or both” (Watson, Gemin, & Ryan, 2008, p. 8). Virtual 
schools vary in design and magnitude from full-time online programs that are statewide to 
supplemental online courses offered in a single district. Online courses differ from the traditional 
classroom in that each course is purchased specifically for the student and therefore the district 
has a direct financial loss when students do not successfully complete a course. This study 
focused on an evaluation of virtual school program components in a north Texas public school 
district. 

Definition of the Problem 

High school students in a North Texas school district have opportunities to take online 
classes for original credit and their success is important to the administration and community. 

The comprehensive evaluation of current practices and supports for students was necessary for 
continual improvement of the online program and increased student success. The district selected 
for this study was a growing school district transitioning from a predominantly suburban district 
to one exhibiting many urban characteristics. Student enrollment over the past 14 years resulted 
in a district growing from 10,000 students to more than 30,000 students, even though the 
enrollment rate the past few years has slowed to approximately 5% per year. The school district 
is located between two major cities in northeast Texas. The total student population of the a 
school district exceeds 32,000 students who are served on 22 elementary school (K-4) campuses, 
six intermediate school (Grades 5 and 6) campuses, six middle school (Grades 7 and 8) 
campuses, and four comprehensive and one career-oriented high school campuses (Mansfield 
ISD, 2011). The ethnic composition for the student population in the school district is 27% 
African American, 21% Hispanic, 45% Anglo, and 7% Asian/Pacific Islander/Other (Texas 
Education Agency [TEA], 2011a). Minority students are the primary student population in the 
district. In addition, 29% of the student population is identified as economically disadvantaged. 

In the school district, recent success data indicate that students taking online courses in a variety 
of subject areas are not performing at levels acceptable to district administration. The courses 
taken by students in the school district included 

• Algebra 1, Geometry, Algebra 2, AP Calculus AB, AP Calculus BC, and Math Models; 

• English 1, English 2, English 3, English 4, AP English Language, and AP English 
Literature; 

• AP Biology, Chemistry, Environmental Science, and AP Environmental Science; 

• World Geography, US History, AP US History, Government, Economics, AP 
Government, and AP Economics; 
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• Spanish 1, Spanish 2, Spanish 3, AP Spanish, French 1, French 3, AP French, German 1, 

German 3, Fatin 1, and Fatin 2 (MISD, 2010b). 

Programmatically, the development of a successful online program benefits students who 
elect to earn credits online. Financially, any online course that is not successfully completed 
costs the district the price of the course and any additional support costs. Online courses offered 
through the Texas Virtual School Network (TxVSN) can cost from $250to$400 for any one- 
semester course depending on the course and the course provider (TxVSN, 2011a). The school 
district spent nearly $25,000 for online classes in which students were not successful during the 
2010/2011 school year. Evaluation of the current practices and supports for students was 
necessary for continual improvement of the online program for the school district. 

The research took place in a North Texas school district and evaluated course curriculum, 
online teachers, course learning management system, campus practices, campus and district 
supports in place for all students taking online courses. The participants in the research project 
included students, teachers, course providers, and administrators actively involved in the school 
district’s online course program. Kirtman (2009) reported that most research related to online 
courses deals with various components of teaching online and is not focused on student success. 
One area of focus for this program evaluation included components of support as well as design. 
Irvin, Hannum, Farmer, de la Varre, and Keane (2009) wrote “little is known about factors that 
contribute to the non-completion of advanced online courses and relatively little research has 
focused on ways to improve course completion and student achievement” (p. 30). In addition, 
little research on the subject has been done for high school students. Therefore, this evaluation 
addressed issues related to high school students’ lack of success in online courses at the local 
level and modified practices within the program may be utilized at the statewide level. 

The rationale for choosing to evaluate the online course program in the North Texas 
school district is that students taking classes online were not successful completing classes at an 
acceptable rate locally or at the state level (TxVSN, 2011c). In 2009, the school district piloted a 
program that allowed high school students to take high school courses online. Educators in Texas 
are increasingly supporting the trend by providing access to online classes for high school 
students. The Texas Education Agency has developed the TxVSN (TxVSN, 2011b), that allows 
Texas high school students enrolled in public or open enrollment charter schools to take courses 
online for high school credit. The program is designed to supplement high school programs but 
does not serve as a pathway to a high school diploma (TxVSN, 2011b). As school budgets 
continue to be cut, nontraditional options are necessary to meet the needs of the 1.3 million high 
school students in Texas (TEA, 2011b) determined to meet the credit requirements for 
graduation. TxVSN is one resource to help meet students' needs in this area. 

A concern associated with the TxVSN program is student success (TxVSN, 201 Id). 

Many school districts take advantage of these online course offerings for students to take courses 
that would otherwise not be possible in their school district. However, enrollment in the online 
course should not be the end of the campus involvement. Schools that enroll students in online 
classes should be aware of the necessary supports for their students and have specifically 
designed local online programs promoting student success. According to statewide data for fall 
2010, more than 35% of the students enrolled in an online class through the TxVSN were not 
successful (TxVSN, 2011c). The local data from the school district were not much better. In the 
first semester of implementation, the student failure or non completion rate in the school district 
was just less than 30%. 
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PURPOSE OF THE STUDY 

This evident of success of the district online program impacts the time invested in the 
new program and the ongoing need to earn credits toward graduation requirements for students. 
Online courses differ from the traditional classroom in that each course is purchased specifically 
for the student and therefore the district has a direct financial loss when students do not 
successfully complete a course. The program itself is impacted because students may become 
less receptive toward taking online classes in the future. The purpose of the study was to evaluate 
which practices and supports of the school district’s online program benefit students and develop 
suggestions to improve the program implemented through a corresponding project (see Appendix 
A). The current practices and supports that were in place were not yielding the expected level of 
success for some students. Evaluating these practices can benefit all students taking online 
courses through the implementation of effective practices and the elimination of ineffective 
practices. 

METHOD 

The goal of this study was to examine whether or not the current online instructional 
program in the school district was effective. To determine if all the components of the current 
program were effective, the need existed to systematically collect data. The method best suited 
for collecting data to determine a program’s worth and make recommendations for programmatic 
refinement and success is a program evaluation (Spaulding, 2008). The evaluation reviewed the 
overall design of the program including the curriculum design, online teachers, learning 
management system, course requirements, current practices of students, teacher mentors, campus 
administrators, and district administrators to determine the most effective practices related to 
student success. The results of the program, both formative and summative, provide guidance to 
the school district’s administration regarding necessary modifications for the program. 

Qualitative Research Design and Approach 

This study utilized a qualitative research design. Qualitative research design best met the 
need of this program evaluation in exploring and understanding the meaning participants place 
on the increased utilization of online learning for high school students (Creswell, 2009). 
Spaulding (2008) described the utilization of participants in evaluating a program as a 
participatory-oriented evaluation approach. This participatory-oriented evaluation approach of 
qualitative research was selected for this project study in order for the researcher to collect data 
regarding the views of the participants through broad, general questions in the form of words and 
descriptions (Creswell, 2008). Within the qualitative framework, a case study research design 
provided the structure for an evaluation of the school district’s virtual school program. Case 
study research design is an in-depth exploration of a bounded system (Creswell, 2008). In this 
case, the bounded system was the school district’s virtual school program. Spaulding (2008) 
defined program evaluation as a study conducted for decision-making purposes. Programs are 
evaluated to determine the value of the program and to provide recommendations for 
improvement. A critical component necessary for this type of study is a program. Spaulding 
(2008) generalized educational programs as programs that take place anytime, anywhere for the 
sake of educating a group. Within a program evaluation, an evaluator studies all the components 
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of the program and provides feedback to the program administrators or organizers. Data are 
collected formatively to make modifications to the program and then summatively to assess the 
impact of the program and the applied modifications. The qualitative design for collecting 
research data was preferred over a quantitative approach in order that data are collected from the 
program participants specifically related to the components of the program. A quantitative 
approach would not provide information related to understanding the participants experiences 
(Creswell, 2008). The evaluation reviewed the overall design of the program including the 
curriculum design, online teachers, learning management system, course requirements, current 
practices of students, teacher mentors, campus administrators, and district administrators to 
determine the most effective practices related to students’ success in online courses. The overall 
goal of the evaluation was to determine the effectiveness of the school district’s virtual school 
program. 

Participants 

The school district’s virtual school program for online courses extended to four 
comprehensive high schools. Students attending these four high schools have the opportunity to 
utilize online course offerings within their course schedules. At the time of the research, student 
enrollment in online courses was just more than 300 students. The high school students actively 
enrolled in online classes for original credit served as the students eligible for selection as 
participants in the program evaluation. The sampling method appropriate for this sample was 
convenience sampling. Creswell (2008) described convenience sampling as when the researcher 
selects participants because they are willing and available. In addition to the students, other 
individuals actively involved with the daily operation of the program; mentor teachers, campus 
counselors, and campus administrators, were included in the evaluation. The number of students 
that participated in the survey conducted by the district was 46 and the number of mentor 
teachers, counselors, and campus administrators was 14. From each high school campus, 18 
students participated in one of four focus groups and five mentor teachers and counselors 
participated in the staff focus group. Gathering information from students willing to participate 
in the program provided a wider spectrum of data for analysis. In order to evaluate the program 
through varying lenses of success, no limits were placed on the available participants based on 
the successful or unsuccessful completion of online courses. A caution discussed by Creswell 
(2008) in using convenience sampling was that the resulting sample may not represent the 
population entirely. 

Data Collection 

Conducting an evaluation within a school district in which professional employment is 
established allowed frequent opportunities to access the students and mentor teachers working in 
the online program. Qualitative data was be collected by conducting survey research through the 
use of questionnaires and interviews. A cross-sectional survey design was utilized for data 
collection. Creswell (2008) described cross-sectional sun’ey design as a study approach by 
which the researcher collects data at one point in time. Creswell (2008) associated the cross- 
sectional survey research method of collecting data with program evaluation because 
participants’ opinions about the practices of the program can be collected. Access to participants 
was coordinated through campus administration. The collection of data through questionnaires 
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was done by the school district through the use of electronic surveys. Data collection involved 
the notification for access and collection of electronic questionnaires utilizing open-ended 
questions to online students, mentor teachers, campus level counselors, and campus level 
administrators. The timeframe for notification and collection of the questionnaires was 1 week. 
Once all questionnaires were completed, the district provided the original electronic versions of 
the questionnaires to the researcher; these questionnaires were copied to an external hard drive 
for security purposes. After qualitative data were collected through questionnaires, interviews 
were conducted in focus groups at each comprehensive high school with randomly selected 
students and one district-wide focus group with mentor teachers and campus staff. These 
interviews were scheduled outside of any students’ regularly scheduled class time and conducted 
by the project study researcher. 

Questionnaire 

Data collection involved the use of a questionnaire. An electronic questionnaire was 
utilized for the distribution and collection of the questionnaire data. As outlined by Creswell 
(2008), the following steps were followed in the development of the questionnaire: 

1. Different types of questions were written. 

2. Strategies for good question construction were utilized. 

3. Validation of the questionnaires. 

Two questionnaires were developed with personal, attitudinal, and behavioral questions; 
sensitive questions; and closed- and open-ended questions (Creswell, 2008) designed to gather 
information related to the online program from the student and campus level administration. 

Each questionnaire consisted of ten questions associated with district and campus levels 
supports, online course curriculum and design, online course learning management system, and 
student performance. The content of the questionnaires was validated through the utilization of 
an expert panel. The expert panel consisted of an online curriculum developer, an online course 
provider administrator, an experienced virtual school administrator, a qualitative research 
methodology expert, and a local school district curriculum expert. The expert panel reviewed the 
questionnaires related to content validity. Content validity was defined by Creswell (2008) as the 
extent in which questions and scores from these questions are representative of all the possible 
questions that could be asked about the content or skills. Once the questionnaires had earned 
acceptance from the expert panel for validity, the questionnaires were posted electronically on 
the district survey website. Notifications for access to the survey were distributed to all online 
students, mentor teachers, campus level counselors, and campus level administrators. The 
information collected from the questionnaires provided thick, rich descriptions related to 
practices perceived as effective or not by the participants in the online program. 

Interviews 

In order to further the detail related to the data collected, focus groups were formed. A 
focus group was assembled at each high school campus with randomly selected campus 
representatives to discuss the different facets of the online program. Interview questions for the 
focus group were developed. These questions were reviewed and modified through the use of the 
previously mentioned expert panel. The resulting focus group questions did not replicate the 
content of the questionnaires, but were written to allow each participant to share his/her 
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experiences in the online program from a personal perspective. Each participant was formally 
invited to participate in the focus group and the focus group interviews took place at each high 
school campus. Each focus group session was recorded and transcribed for data analysis. During 
the focus group interviews, the interviewer recorded notes related to the reactions observed and 
additional information thought to be necessary (Creswell, 2008). 

Data Analysis 

Questionnaire and interview data were analyzed, coded, themes developed, and themes 
interpreted for meaning. Creswell (2008) emphasized with qualitative research, many 
components of data collection, analysis, and reporting are simultaneous activities and researchers 
must cycle back and forth between these phases. Creswell (2008) reported the importance of 
reading through the transcripts many times in order for the researcher to immerse himself/herself 
in the details and get a sense of the resulting data before organizing information into smaller 
components. The transparency of the research process with qualitative research method is critical 
for the trustworthiness of the data analysis (Ryan, 2009). All questionnaire data were received 
from the school district and saved electronically in order to preserve the data collected. The 
information obtained from the questionnaires was imported into a qualitative data analysis 
software program. The comments provided from the questionnaires were entered into the 
software. Each focus group transcript was uploaded into the software. QSR NVivo 10 was the 
qualitative analysis software utilized for data organization and analysis. 

The organization of the data began with grouping like responses or text segments 
(Creswell, 2008). The next step identified by Creswell (2008) in the qualitative process that was 
accomplished was coding the data. Coding is the process of segmenting and labeling collected 
data into broad themes (Creswell, 2008). High school students’ opinions related to beneficial 
practices of the Virtual School Program may be impacted by their academic performances in the 
online classes. Therefore, discrepant cases may reflect extremely negative feedback. Through the 
coding process, any discrepant cases were noted and evaluated independently as information that 
added value to the project study. Any discrepant case was not automatically discarded but 
reviewed for value. Open-ended questions in the questionnaire were utilized to gather data from 
students and staff and codes assigned to different groups of text segments. The codes were 
described as major topics, unique topics, and leftovers (Creswell, 2009). From this analysis of 
topics utilized, the most descriptive wording (Creswell, 2009) was used in developing themes. 
Themes, defined by Creswell (2008), are similar codes organized into major ideas. The resulting 
themes were organized into layered themes that represent major and minor developed themes. 
The final themes are being reported as the findings of the project study. 

RESULTS 

The complete set of data collected included student and faculty survey results, comments 
from these surveys, transcribed focus group data from the four student focus groups, and 
transcribed data from the faculty focus group. During the data analysis, these data were read and 
reread to develop an overall understanding of the participants’ views associated with their 
responses. The resulting themes developed through the qualitative data analysis were support, 
online courses, and technology. Each of these areas is presented using data collected from the 
questionnaires and focus groups for both students and faculty. The questionnaire data collected 
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from online students provided incite to their perceptions of effective supports currently being 
implemented in the school district’s virtual school program in three major areas: campus level 
support, district level support, and online support. From the questionnaire data, students 
identified two effective campus level supports from their online experiences as computer access 
and scheduling. The priority for students was the ability to get in a location during the school day 
in which they could access the online class. Findings from the student focus groups related to 
course access will be discussed within the technology theme. In addition two ineffective campus 
level supports were progress reports and tutoring. Questionnaire data also revealed that students 
felt a need for additional support at the campus level in the area of subject specific teacher 
availability or tutoring. The new experience for students not having a face-to-face teacher to ask 
a question when questions arose was a concern. Adjusting to not having a teacher was 
emphasized best by a student commenting, “Campus supports were excellent with the exception 
of not having an available teacher in the subject matter.” Students are forced to seek other means 
of subject specific support other than the teacher in the classroom. 

The campus supports perceived by the faculty that took the survey as effective were 
computer access, scheduling, and teacher mentors. These results mirrored the responses from the 
students. Faculty members emphasized the importance of students having access to computers at 
school to login to their online classes. Scheduling of students was effective but some faculty 
identified the counselors as the sole individuals responsible for this task. The individual effort of 
individual mentor teachers was a focus of a few faculty responses. The role of teacher mentor 
was clearly identified by faculty as an effective support. However, comments related to this 
support identified many areas of concern for mentor teachers such as: access students’ progress; 
more knowledge about the learning management system used for the courses; knowledge about 
academic supports available for students; how to access the online teacher; and other program 
related suggestions. The faculty focus group also emphasized mentor teachers as an effective part 
of the program and emphasized the need for more tools to support students. One faculty member 
stated, “The district needs to communicate what tools are available for mentors that are built into 
the system.” An ineffective campus support identified by the faculty was technical support. 
Faculty members working with the program expressed a lack of technical support from the 
campus. One faculty member stated, “Most of the time we would just troubleshoot issues without 
technical support.” Technical support is an integral part of a successful program for students to 
take classes online. 

The second area of support identified in the student questionnaire was district level 
support. The students identified two effective district level supports as progress reports and 
technical support. Two ineffective district level supports were progress reports and parent 
communication. Interestingly, students felt strongly about the effectiveness of progress reports in 
both a positive and negative manner. The issue of technical support was considered effective 
while students considered on-campus access to needed materials a district level concern. 

The district level supports perceived as effective by faculty were progress reports and 
technical support. This also mirrored the effective supports identified by students. An ineffective 
district level support identified by the faculty was parent communication. Some faculty 
commented that they were not aware of any parent communication that was done at the district 
level beyond three and six week progress reports that go out to all students and that they enter the 
grades. Additional district level supports recommended by the faculty focused predominantly on 
training for the mentor teachers. 
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The third area of support identified in the student questionnaire was online support. 
Students identified two effective online supports were email and technical support. Two 
ineffective online supports identified by students were discussion board and email. As observed 
previously, students felt strongly about both the effectiveness and ineffectiveness of course 
email. At least one student did not feel as much communication with the online teacher was 
necessary, noting, “If you have questions on the homework, you do not have to ask a teacher 
because you can go back in the course and look up the answer.” Students’ experiences related to 
email directly relate to comments provided by students related to their ability to communicate 
with their online teacher. Frequent comments from the questionnaire suggested a need for more 
feedback from online teachers. Students expressed concerns related to having multiple online 
teachers for a course leading to inconsistent feedback. The timeliness of the feedback was also a 
concern. Some students still sought a one-to-one relationship with the online teachers. For 
example one student commented, “I think that each student should be assigned to a SINGFE 
teacher, (like a classroom) so that if we need help we know exactly who to email for help in a 
course. Besides I'd like to know who was grading my assignments, the one-on-one with a teacher 
is important.” More effort must be made in developing a positive relationship between online 
teacher and student in the virtual environment. 

The faculty component of online support confirmed the student view in identifying online 
technical support as an effective online support. One faculty member stated, “...the online 
teachers were always very willing to help if a student needed assistance.” However, faculty also 
identified online technical support, along with live chat and phone support as ineffective online 
supports. The ineffective online supports were the focus of comments and suggestions from the 
data collected. At least one faculty respondent felt that the use of almost all online supports 
resolved very few of students’ issues and resulted in solutions developed at the campus or 
classroom level. The most common online resource requested by teachers was additional or 
updated online resources aligned with the class. One teacher commented, “Some courses need to 
have their websites updated with current assignments and grades.” The teachers expected the 
online class assignments to be up-to-date and grades reported frequently. 

Other areas identified by students in the area of support were enrollment and online 
teachers. The enrollment process for students did not seem to be a major issue except for 
promotion of the program. More than one student expressed a surprise that online classes were 
not promoted more on their campus. Online teachers serve as a major component of the online 
environment. The online teacher component of online courses was the focus of the outlier data 
collected. Students frustrated with the concept of online learning focused on the absence of the 
availability of a face-to-face instructor when communicating their frustrations. One student 
communicated a lacking component when stating, “Face-to-face communications so that the 
teachers can't hide and the students can actually participate. This type of class is way too anti¬ 
social.” Students expressed frustration with online classes in general because of the lack of a 
teacher physically present with them. 

The second theme developed from the data collected from student and faculty responses 
was online courses. The significant components within the theme of online courses included 
pace, course design, and course content. Students expressed in an emphatic manner that the 
ability to work at their own pace and complete classes at an accelerated rate were the most 
successful components of the online program. Pace also encompassed the ability to work on a 
class anytime, anywhere or flexibility for students. The phrase, “just being able to go at my own 
pace” was stated by many students. Another student stated, “Some classes go at a slow pace and 
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it feels it will take an eternity to get it done. This class works the way it fits for me.” Advice 
provided from online students emphasized pace when they stated, “Future students need to pay 
attention to the pacing guide of a course and not to procrastinate. Don’t think just because I can 
do this at my own pace, it’s not really. I don’t have to work this day because I can just go on and 
do it at home. Every day you should be able to come in and work.” “Being able to finish at my 
own pace, not having to wait for teachers to lecture about something. With online you are able to 
work as fast as you want or take your time if you want.” Pace affected whether or not students 
were willing to take online classes in the future. One student revealed, “I liked it better than face- 
to-face instruction because you can choose when to do it and when not to do it.” The faculty 
response to pace also mentioned student success in courses sooner than required in traditional 
semester long courses. The discussion evolved to pace serving as positive motivation for 
students. One faculty stated, “In online classes there is an incentive for students to move forward 
whereas in the classroom, students have to follow a certain pace.” Both online students and 
faculty working with the virtual school program viewed working at a flexible pace as one of the 
greatest benefits of taking an online class. 

The second component developed from the online course theme was course design. Two 
effective features of the course design identified by students were course information and course 
checklist. Both of these features provided students tools to best navigate their online courses 
successfully. Students compared the online course design to their experiences in traditional face- 
to-face courses when commenting, “It is kind of like a real class, just because it is online it not 
easy. It is a real class. The setup is different but what you learn in the class is no different. Not 
blow off easy, but since it was a high school course I thought it would be easier. I thought that 
before I started doing it. It still requires you to pay attention to it.” Another student responded 
about course design by saying, “I would say just how effective it is. You absorb the information 
because you can go back and read it and read it and read it. And before a test you can go back to 
the lesson and read it and read it. It is just as easy to understand as most classes I have taken.” 

An ineffective feature of the course design identified by students was resource library/external 
links. The frustration most identified related to this feature was that links were out-of-date or 
could not be accessed through the school district network. Many of these links were to YouTube 
or other social media sites which the school district limited access. 

Faculty responses related to effective course design features resulted in favorable 
opinions about most of the features. One feature identified as effective was talk to teacher. The 
talk to teacher feature allowed students to live chat with an online teacher. Comparatively, 
students identified talk to teacher as an ineffective feature. This discrepancy may be an issue of 
student awareness regarding the feature. One mentor teacher stated, “This feature was split. 

Some students found it beneficial and others needed hands-on assistance.” Additional features of 
the online environment discussed by faculty members as positive features were having a record 
of everything a student does and the ability to communicate with the online teacher directly. 
Mentor teachers favor being able to login and view exactly how much time a student has spent 
working on the online course. Faculty responses identified course checklist as one ineffective 
feature of the online course design. This result also contradicts the results from the students. A 
mentor teacher commented, “Students do not follow this list.” Another teacher felt that some 
course checklists needed updating within the course websites. A better working knowledge of the 
course design features by the faculty could be a reason for this difference. 

The third component developed from the online course theme was course content. Two 
effective features identified by students within course content were course checklist and grade 
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book/progress reports. Again, these features give students information critical to daily activities. 
Other positive comments related to of the online courses referenced the pacing progress bar as 
helpful and many students recommended using the course calendar. Other effective features 
expressed by students included class notes and instructions for assignments. One student 
commented, “The course gave you the notes to study before the test.” Another student compared 
the written instructions provided online to her experience in the traditional classroom by saying, 
“Whenever I would take a test it gave me a set of instructions. A teacher would not give you 
every detail if you needed to write an essay and repeat it over and over. Here it is all written out.” 
Another student taking a core course online also commented about the differences, “English is 
one of my weakest subjects and it was nice to take it online with the same material in the same 
class but the setup and everything was different. The course would have review sheets before the 
test to help study. Able to go back and scan material not remembered on the review. Good 
balance between tests and papers.” The three least effective features within course content were 
the orientation course, instructor biography, and discussion board. The comments and focus 
group discussions referenced communication components within the course content as important. 
Students recommended increased use of face-to-face discussions or chats to improve 
communication and the use of videos for instructional purposes. Another negative aspect related 
to course content was that all the necessary resources were not included as part of the course. 
Students were directed to search Google for the material. Concerns associated with the course 
content of a specific music theory course included lacking actual audio. This student stated, 
“There were a few samples, but I thought part of the course would be hearing what its sounds 
like and what the differences between each part.” The continual development of more interactive 
features in online classes will help students be more engaged. 

The corresponding results from faculty surveys identified the effective features of online 
course content were grade book/progress reports, course calendar, resource library/ external 
links, course calendar, course email, and talk to teacher. The consistent component when 
comparing to students results was grade book/progress reports. Faculty working with the 
program did not view course checklist as an effective part of the courses as students did. 
Ineffective features of the online courses as reported by the faculty were course calendar, 
announcements, course checklist, and talk to teacher. These features are all major components of 
each online class and faculty members’ experiences in viewing these components may have been 
lacking. For example, one faculty member commented in the survey that she had never seen a 
course discussion board, instructor biography, course information, or announcements. Another 
area that surfaced during focus group interviews related to course syllabi. One faculty member 
shared her experience by saying, “I think in the course literature they would say to go to a certain 
page in a certain book and the syllabus did not mention the book. The syllabus was not always 
what they needed. It was not matching.” This was confirmed by another who stated, “The 
content of the syllabus needs to be accurate.” The expectations were that materials are up-to-date 
and accurate. 

The third theme that evolved from the data collected from student and faculty responses 
was technology. Within the technology theme, two major components developed 
access/connectivity and program design. The overwhelming concern associated with 
access/connectivity was exactly that, network access to the online course. Any issue that would 
not allow a student to connect through the Internet to their online classes was identified. The 
issues identified included local network problems, course provider problems, and compatibility 
problems. Anytime students would experience a slow school network or could not access certain 
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content because of filters, progress toward success was halted. On student commented, “A faster 
communication online is needed because sometimes it is not efficient.” Beyond a slow network, 
school filters created frustration for students. One student expressed his frustration when stating, 
“I needed ways to access needed materials at school that might be blocked, not simply told I 
need to do the assignment at home.” Students also discussed concerns associated with technical 
difficulties on the course provider side. Students reported that on occasion a course would freeze 
or not allow a student to finish the desired assignment. One student stated, “It was frustrating 
when I got kicked out of a test by clicking the back button. It took all day to get caught up.” 
Another student felt that progress was impacted by technical issues when stating, “I kept getting 
logged out and I had to chat with the teacher to get everything reset. I felt like I got behind.” The 
last area of concern expressed by students associated with accessibility dealt with compatibility. 
Students working on their online classes using iPads discovered some limitations. One student 
said, “When on the iPad, I cannot do the word processing for the course.” Another commented, 
“Some video and links to videos did not work on the iPad or would not play. Some videos would 
only allow you to view once. Once you moved on, you could not go back. When I tried to view it 
at home I could not find it, but when I tried again on the iPad it would not work.” Learning to 
work in an online class using new technology was sometimes frustrating for students. 

Faculty responses related to access mirrored those of students emphasizing district 
imposed limitations to certain resource links. One emphasized, “A frustration is when the course 
directs a student somewhere, like go the ecological footprint and it does not work on our 
computers. Some of the class was not compatible or the filter blocked it.” Faculty members 
further discussed some local program design concerns associated with technology use. The 
mentor teachers expressed a concern with knowing how to access students’ courses, being 
familiar with the course contents, and having the ability to access student success data without 
depending on someone else to retrieve the data. One suggested, “Show mentor teachers what 
students are going to see from time to time, the different screens, and how to talk with a teacher.” 
Teachers working with the virtual school program wanted to be experts in order to maximize 
their ability to assist students. 

CONCLUSIONS 

This study systemically evaluated a north Texas school district’s virtual school program 
in order to provide guidance for collecting, interpreting, and reporting data to improve the 
program. The school district’s virtual school program was researched utilizing a qualitative 
program evaluation. Data were collected through the use of a cross-sectional survey design 
utilizing a questionnaire and focus group interviews. The resulting data analysis determined both 
effective and ineffective practices of the program as perceived by the participants in the program. 
The results revealed that mentor teachers were not aware of all the different aspects of the 
students’ online courses and needed additional tools to better assist students. Providing a 
comprehensive professional development program for all mentor teachers allows students to 
have a viable resource for needed assistance. The potential impact this could have on students is 
an increased ability to be successful because of the increased effectiveness of mentor teachers. 
This program modification related to mentor teacher professional development, which developed 
from the results of an evaluation of the school district’s virtual school program, will positively 
impact all future students in the school district by providing viable options to the face-to-face, 
brick and mortar school setting that currently dominates the public school setting. The increased 
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utilization of online learning for high school students has the potential to change the way that 
administrators, teachers, students, and parents view the system of school in the future. 
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